Around the first bunch of postings in my Useful but Unknown Unix Tools, Tobias Klauser of inotail and Symlink fame came up with the idea of making a Planet (i.e. a blog aggregator) of all the comandline blogs and blog categories out there. A first Planet Venus running prototype based on the template and style sheets of Planet Symlink was quickly up and running. I just couldn t decide if I should use an amber or phosphor green style for this new planet. Marius Rieder finally had the right idea to solve this dilemma: Offer both, an amber and a phosphor green style. Christian Herzog pointed me to the right piece of code at A List Apart. So here is it, available in you favourite screen colors:

Planet Commandline For a beginning, the following feeds are included:

• Myon s Unix blog category
• Evgeni Golov s Desktop in a Shell column
• My Useful but Unknown Unix Tools column
• My Shell blog category
• The GRML Development Blog
• Chr tertee (Swiss-German)
• Commandline Magic s identi.ca feed

Which leads us to the discussion what kind of feeds should be included in Planet Commandline. Of course, all blogs or blog categories which (nearly) solely post neat tips and tricks about the command line in English are welcome. Microblogging feeds containing (only) small but useful command line tips are welcome, too, if they neither permanently contain dozens of posts per day nor have a low signal-to-noise ratio. Unfortunately most identi.ca groups do, so they re not suitable for such a planet. What I m though unsure about are non-English feeds. Yes, there s one in already, but I noticed this only after including Beat s Chr tertee and his FreeBSD command line tips are really good. So if it doesn t go overboard, I think it s ok. If there are too many non-English feeds, I ll probably split Planet Commandline off into at least three Planets: One with all feeds, one with English only and one with all non-English feeds or maybe even one feed per language. But for now that s still a long way off. Another thing I m unsure about are more propgram specific blogs like the impressive Mastering Emacs blog about mastering the world s best text editor . *g* (Yeah, I didn t include that one yet. But as soon someone shows me the vi-equivalent of that blog, I ll include both. Anyone thinks, spf13 s vim category is up to that?) Oh, and sure, any shell-specific (zsh, tcsh, bash, mksh, busybox) tips & tricks blogs don t count as program-specific blogs like some $EDITOR, $BROWSER, or $VCS specific blogs do. :-) Of course I m happy about further suggestions for feeds to include in Planet Commandline. Just remember that the feed should provide (at least nearly) exclusively command line tips, tricks or howtos. Suggestions for links to other commandline related planets are welcome, too.

Using a virtual IP managed by heartbeat is a nice way to work around the slow fallback behavior with multiple nameservers in /etc/hosts. But unlike ntpd, bind9 doesn't automatically listen on new IPs on the system. Here's a cute hack to fix that:

# cat /etc/ha.d/haresources
server01 bind9release IPaddr::10.0.0.3 bind9takeover
# cat /etc/ha.d/resource.d/bind9release
#!/bin/sh
# when giving up resources, reload bind9
case $1 in
        stop) /etc/init.d/bind9 reload ;;
esac
exit 0
# cat /etc/ha.d/resource.d/bind9takeover
#!/bin/sh
# on takeover, reload bind9
case $1 in
        start) /etc/init.d/bind9 reload ;;
esac
exit 0

When my wife bought her HP 2605dn color laser, she had also ordered a set of replacement cartridges because the originals would supposedly not last very long. It turned out that they would last over two years, though. Now, the black cartridge was empty. The replacements are original HP components, but recycled/refilled second hand from a different source. I put in the black one, and the trouble started with the new drum having a big scratch that would appear in every print. After two years, returning the cartridge was not an option. As the old drum was still good, the idea was to put the new toner into the old cartridge. A bit of googling (and visual inspection) quickly revealed that you are not supposed to do this, the device doesn't have any facilities to access anything. Some post did indicate though that the cartridge actually consists of two parts that are only held together by two pivot pins and two small springs. If you pull the two parts apart a bit, you can feel the axis the parts are moving around. So, I would try to use the old drum part with the new toner container part. Removing the springs is easy (also removing the flap covering the drum, along with a tiny third spring). The two pivot pins are pretty inaccessible, but I managed to pull out both using a pointy tong. One pin is made of metal, if you need more space you can drill a small hole right next to it. The other pin is made of plastic with a small hole in it, in one case I succeeded by screwing in a little screw and pulling at it. Reassembling the parts is not so hard. I put the "new" cartridge into the printer, waited for the initialization to finish, only to find out the printer would still think the cartridge was empty and should be replaced. Duh. I then discovered a small chip attached to one corner of the drum unit, probably carrying product ID and some serial number. Luckily, it was easy to be removed and exchanged with the other drum unit. Still, the printer thought the cartridge was empty. Some (windows) driver re-installations later it was clear that there is no "ignore fill level and print anyway" option. (Cups didn't print either.) The HP Support Forum has a long Ink cartridge empty but it's not thread with lots of complaints but few answers. (Their problem is mostly with new HP cartridges becoming "empty" after a few days.) Someone suggests trying to cold reset the printer. For the 2605dn, that's holding the left and right buttons while powering on. I removed the cartridges for doing so. When I had put them back in, the printer asked me to confirm that non-HP material was installed, and subsequently didn't display any fill level for the black cartridge anymore, just " ? " in the display. Printing works now. Yay. (Lesson learned: I still hate dealing with printers.)

Jan 5 2011, 3070g, 54cm Julia, Mom and Dad are all well and very happy.

Mutt's configuration is sometime more a toolbox than something offering ready solutions, and "how to I use multiple accounts?" is one of the most FAQs. Here's a condensed version of my setup. In the most simple solution, just 'c'hange folders to the IMAP server:

c imaps://imap.example.com <enter>
c imaps://imap.otherdomain.tld <enter>

That's cumbersome to type, so let's automate it:

# .mutt/muttrc
macro index <f2> '<change-folder>imaps://imap.example.com<enter>'
macro index <f3> '<change-folder>imaps://imap.otherdomain.tld<enter>'

That would be the basic setup. The two accounts have settings associated with them, we put them in two files:

# .mutt/account.example
set from=me@example.com
set hostname="example.com"
set folder="imaps://imap.example.com/"
set postponed="=Drafts"
set spoolfile="imaps://imap.example.com/INBOX"
set signature="~/.mutt/signature.example"
# .mutt/account.otherdomain
set from=myself@otherdomain.tld
set hostname="otherdomain.tld"
set folder="imaps://imap.otherdomain.tld/"
set postponed="=Drafts"
set spoolfile="imaps://imap.otherdomain.tld/INBOX"
set signature="~/.mutt/signature.otherdomain"

Now all that's left to do is two folder-hooks to load the files:

# .mutt/muttrc
folder-hook 'example.com' 'source ~/.mutt/account.example'
folder-hook 'otherdomain.tld' 'source ~/.mutt/account.otherdomain'
# switch to default account on startup
source ~/.mutt/account.example

A slight variation of the macros also uses the account files:

macro index <f2> '<sync-mailbox><enter-command>source ~/.mutt/account.example<enter><change-folder>!<enter>'
macro index <f3> '<sync-mailbox><enter-command>source ~/.mutt/account.otherdomain<enter><change-folder>!<enter>'

To save entering the password all the time, we use account-hooks:

account-hook example.org 'set imap_user=me imap_pass=pw1'
account-hook otherdomain.tld 'set imap_user=myself imap_pass=pw2'

Putting passwords in configs isn't something I like, so I pull them from the Gnome keyring:

set my_pw_example= gnome-keyring-query get mutt_example 
set my_pw_otherdomain= gnome-keyring-query get mutt_otherdomain 
account-hook example.org 'set imap_user=me imap_pass=$my_pw_example'
account-hook otherdomain.tld 'set imap_user=myself imap_pass=$my_pw_otherdomain'

(I found gnome-keyring-query in the Gentoo Wiki.) Martti Rahkila has more verbose article with similar ideas.

Most of my talks are either talks about commandline basics (Commandline Helpers about GNU Coreutils and Findutils usage; Understanding Shell Quoting) or collections of small tips and tricks, e.g. about SSH, History Expansion, or commandline tools. Valentin Haenel called this second type of talks nugget talks . I like that term. My newest talk (German only so far, next occasion at the LUGS meeting in Zurich on 16th of December 2010) is such a nugget talk and is about useful but unknown tools. While preparing that talk I noticed that the amount of neat but less known tools I collected for possible inclusion into that talk would be enough for a whole day or so. I noticed that I ll never be able to give a talk about all that stuff. To not letting these tips rot in that file uncommented, I decided that I ll write (more or less) short blog postings about these tools, similar to and inspired by Myon s blog postings about Cool Unix Features as well as the no more existing or at least broken Debian Package of the Day and Debian Package a Day. While Myon mostly posted long-time but still unknown features, my postings will mostly cover useful but unknown packages available in Debian and most debian-derived distributions like grml and Ubuntu. Some postings may also just highlight some features of common tools or packages like e.g. GNU Coreutils. You can subscribe to these tips via RSS or just view all of them in your web browser at http://noone.org/blog/English/Computer/Debian/CoolTools/.

I need to look this up every time I need a backport (mostly PostgreSQL) at a customer site with limited networking:

$ lftp -c 'mget http://backports.debian.org/debian-backports/pool/main/p/postgresql-8.4/*_8.4.5-1~bpo50+1_amd64.deb'

Hopefully I can remember this in the future.

I need to look up this every time I need a backport (mostly PostgreSQL) at a customer site with limited networking:

$ lftp -c 'mget http://backports.debian.org/debian-backports/pool/main/p/postgresql-8.4/*_8.4.5-1~bpo50+1_amd64.deb'

Hopefully I can remember that in the future.

A few months after starting the NM process, I have just been accepted as a Debian Developer. My account name is simply: julien I have been a Debian user for about 10 years now, and have begun contributing to Debian in 2005. I have then been accepted as a Debian Maintainer in 2007. This post is mainly to thank:

• all my sponsors: Christoph Haas (haas) , Lo c Minier (lool), Free Ekanayaka (freee), Maximiliano Curia (maxy) and Micah Anderson (micah),
• my advocate: Matthijs M hlman (matthijs)
• my AM (who actually make it happen in a really short time frame, thanks a lot to him): Xavier Oswald (xoswald)
• all people who helped me during the process, to name a few: Christoph Berg (myon), Enrio Zini (enrico), Peter Palfrader (weasel)

Also thanks to all people who have already sent their congratulations, it makes me very proud!

/etc/fstab files tend to be an unreadable mess of unaligned fields.

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/mapper/benz-root /               ext3    errors=remount-ro 0       1
/dev/sda1       /boot           ext3    defaults        0       2
/dev/mapper/benz-home /home           ext3    defaults        0       2
/dev/mapper/benz-swap_1 none            swap    sw              0       0
newton:/home        /nfs        nfs defaults,soft,intr,users    0 0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

Let's remove some whitespace in the third line:

#<filesystem> <mountpoint>   <type>  <options>       <dump>  <pass>

And then pipe everything from line 3 to the end through column -t:

# /etc/fstab: static file system information.
#
#<filesystem>            <mountpoint>   <type>       <options>                 <dump>  <pass>
proc                     /proc          proc         defaults                  0       0
/dev/mapper/benz-root    /              ext3         errors=remount-ro         0       1
/dev/sda1                /boot          ext3         defaults                  0       2
/dev/mapper/benz-home    /home          ext3         defaults                  0       2
/dev/mapper/benz-swap_1  none           swap         sw                        0       0
newton:/home             /nfs           nfs          defaults,soft,intr,users  0       0
/dev/scd0                /media/cdrom0  udf,iso9660  user,noauto               0       0

Thanks to SP8472 for bringing this to my attention.

Just discovered (thanks to XTaran): df -T -- show file system type.

$ df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/mapper/benz-root
              ext3    6.6G  4.9G  1.4G  79% /
tmpfs        tmpfs    1.6G     0  1.6G   0% /lib/init/rw
udev         tmpfs    1.6G  228K  1.6G   1% /dev
tmpfs        tmpfs    1.6G     0  1.6G   0% /dev/shm
/dev/sda1     ext3    236M   21M  203M  10% /boot
/dev/mapper/benz-home
              ext3    135G   82G   53G  61% /home
newton:/home   nfs    459G  145G  291G  34% /nfs

paste is one of those tools nobody uses [1]. It puts two file side by side, line by line. One application for this came up today where some tool was called for several files at once and would spit out one line by file, but unfortunately not including the filename.

$ paste <(ls *.rpm) <(ls *.rpm   xargs -r rpm -q --queryformat '% name  \n' -p)

[1] See "J" in The ABCs of Unix

$ for i in  seq 1 40  ; do ./something $i ; done
$ for i in $(seq 1 40) ; do ./something $i ; done

seq is nice for that, but the syntax feels a bit hard to type. If you don't need sh compatibility (read: in bash/zsh), try:

$ for i in  1..40  ; do ./something $i ; done

PS: no spaces allowed PS 2: another useful feature is prefixes/suffixes as in "touch 1..40 .txt"

When there a security update, just logging in to every host and issuing "apt-get install $pkg" doesn't work as the package might not be installed there. The fact that apt-get doesn't understand "apt-get upgrade $pkg" has bugged me for a long time. Recent aptitude versions support that, but that's not part of Lenny. Here's a shell function that does the trick:

upgrade ()  
        if [ "$*" ] ; then
                set -- $(dpkg -l "$@"   grep ^ii   awk '  print $2  ')
                if [ "$*" ] ; then
                        echo "apt-get install $@"
                        sudo apt-get install "$@"
                else
                        echo "Nothing to upgrade"
                fi
        else
                sudo apt-get upgrade
        fi
 

One application is upgrading a lot of hosts when logged in with clusterssh.

I'm a fan of LC_MESSAGES=de_DE and localized web pages -- almost. Unfortunately, the wording of many translations in Debian often drives me nuts. This applies mostly to web pages and package descriptions. Apologies to all hard-working translators -- the percentage of good translations does not outweigh the "wtf" cases for me. Still, I didn't want to switch to English for all the web. Rhonda pointed me to a Firefox ^W Iceweasel plugin: HeaderControl. Now *.debian.org is English for me (reading the constitution in German doesn't solve DAM problems anyway). Yay!

Sometimes remote accounts are only reachable with a series of ssh/su/sudo commands. Using ProxyCommands in .ssh/config works for simple cases, but not with several hops, or if passwords have to be entered. The belier tool takes as input a series of user@hostname strings and will produce an expect script that does the actual login work.

$ cat input
user@host1 root pw1
user2@host2 pw2
$ bel -e input
$ cat host2.sh
#!/usr/bin/expect -f
set timeout 10
spawn ssh -o NoHostAuthenticationForLocalhost=yes -o StrictHostKeyChecking=no  user@host1
expect -re  "(% # \\$) $"
send -- "su - root\r"
expect ":"
send -- "pw1\r"
expect -re  "(% # \\$) $"
send -- "ssh -o NoHostAuthenticationForLocalhost=yes -o StrictHostKeyChecking=no  user2@host2\r"
expect -re  @[^\n]*: 
send -- "pw2\r"
expect -re  "(% # \\$) $"
interact +++ return

The generated host2.sh script uses ugly ssh options, but is easily edited. Now I need the same thing for scp... Update: Carl Chenet, belier's author, kindly pointed me to the documentation which has examples how belier can set up ssh tunnels to copy files.

Yesterday (Monday), Laura and I got married.

I've always been annoyed about how hard it is to convert seconds-since-epoch to strings. I've always been using "date -d '1970-01-01 + 1234 sec'", but as it turned out, that's wrong because it uses the wrong timezone. Luckily, there's a slick replacement:

$ date -d '@1234'
Do 1. Jan 01:20:34 CET 1970

The right version of the "long" version is:

$ date -d '1970-01-01 UTC + 1234 sec'
Do 1. Jan 01:20:34 CET 1970

Since coreutils 8.1 (in Squeeze, not Lenny), there is a command that simply prints out how many processors (cores, processing units) are available:

$ nproc
2

The use case is obvious:

$ make -j $(nproc)

On a side note, this is gnulib's nproc module wrapped into a C program. If you didn't know gnulib before (it had slipped my attention until recently), it is a library of portability functions and other useful things. Adding it to a project is simply done by calling gnulib-tool and tweaking a few lines in the automake/whatever build scripts. PS: do not use nproc unconditionally in debian/rules. Parse DEB_BUILD_OPTIONS for "parallel" instead.

Lockfiles are usually hard to get right, especially in sh scripts. The best bet so far was "lockfile" included with procmail, but its semantics are pretty weird. (Try to understand what exit code you want, with or without "-!".) Not to mention that failing to clean up the lockfile will stop the next cron run, etc. Since Lenny, util-linux ships "flock". Now you simply say

$ flock /path/to/lockfile /path/to/command

and are done. If you want it non-blocking, add "-n":

$ flock -n /path/to/lockfile /path/to/command

I should probably migrate all my cronjobs to use this.